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1.0  EXECUTIVE  SUMMARY 


Historical  aerial  photography  is  routinely  used  in  support  of  environmental  cleanup  operations  at 
many  Department  of  Defense  (DoD)  sites.  The  broader  use  of  current  best  practices  for  image 
interpretation  and  the  application  of  digital  image  processing  (DIP)  techniques  can  improve  the 
amount  of  information  extracted  from  the  photos.  This  appears  especially  important  at  large 
Formerly  Used  Defense  Sites  (FUDS)  that  were  used  as  practice  and  demolition  bombing  ranges 
(DBR)  for  bombardier  training  crews  during  World  War  II  (WWII).  The  accurate  mapping  of 
range  features  and  the  identification  of  sites  where  demolition  ordnance  was  used  is  critical  to 
the  cost-effective  environmental  cleanup  and  transfer  of  these  properties  to  other  public  and 
private  uses. 

The  goal  of  this  project  was  to  demonstrate  techniques  to  make  more  effective  use  of  historical 
photography  for  environmental  cleanup  support.  Eight  study  sites  located  in  the  Southwestern 
portion  of  the  United  States  were  selected  for  the  demonstration  project.  Six  of  the  sites  were 
located  in  New  Mexico  and  two  sites  were  located  in  Texas.  The  sites  provided  a  wide  range  of 
sizes,  spanning  from  1  to  774  square  miles. 

A  comparative  analysis  of  three  interpretation  methods  was  undertaken.  The  baseline  method 
used  existing  results  from  photo  interpretations  (PI)  used  to  develop  Archive  Search  Reports 
(ASR)  for  Military  Munitions  Response  Program  (MMRP)  sites.  These  interpretations  were 
based  on  photographic  prints  and  pocket  stereoscopes.  The  second  method  was  based  on  film 
diapositives  (positive  transparencies)  and  zoom  stereoscopes.  The  third  method  was  based  on 
scanned  film  that  was  digitally  processed  and  viewed.  Both  alternative  methods  improved  upon 
the  baseline  method,  with  the  digital  processing  approach  systematically  providing  the  best 
results.  It  was  observed  by  the  image  analysts  that  routine  image  enhancement  techniques  may 
provide  results  similar  to  the  more  advanced  image  restoration  techniques  tested. 

Some  observations  and  lessons  learned  from  the  demonstration  were  largely  tangent  to  the 
primary  objectives  of  the  project.  One  observation  was  that  the  all  roads  should  be  mapped  and 
considered  for  their  potential  use  as  historical  convoy  targets.  These  targets  appear  to  have  been 
transient  with  no  distinguishing  appearances.  The  presence  of  service  roads  to  nearly  all  targets 
was  noted.  The  historical  documents  mentioned  the  need  to  develop  roads  for  periodic 
maintenance  of  the  targets. 

It  was  recognized  during  the  study  that  aerial  photography  archives  are  dynamic.  Collections 
continue  to  expand  and  they  are  becoming  more  organized  and  accessible  with  improved  finding 
aids.  ASR  photo  search  results  should  be  used  as  a  starting  point  for  subsequent  site 
investigations  (SI)  and  remedial  efforts,  but  the  photo  searches  sometimes  need  to  be  updated.  A 
substantial  amount  of  additional  aerial  photography  was  identified  for  the  sites  used  for  this 
demonstration  project.  It  allowed  the  identification  of  several  ranges  that  had  been  missed  or 
incorrectly  located  in  previous  studies.  Although  many  range  features  remain  apparent  on  recent 
photography,  some  features  were  no  longer  evident  in  as  little  as  four  years  after  range 
operations  were  discontinued.  Acquiring  historical  photography  during  or  shortly  after  range 
operational  periods  can  be  critical  to  the  accurate  detection  and  mapping  of  range  features. 
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2.0  INTRODUCTION 


Historical  aerial  photography  is  routinely  used  in  support  of  environmental  cleanup  operations  at 
DoD  sites.  For  large  WWII  sites  with  a  history  of  military  munitions  use,  such  as  practice  and 
DBRs,  historical  aerial  photography  can  provide  a  unique  source  of  information.  The  accurate 
mapping  of  range  target  features  is  critical  to  the  cost-effective  environmental  cleanup  and 
transfer  of  these  properties  to  other  public  and  private  uses.  The  broader  use  of  current  best 
practices  for  image  interpretation  and  the  application  of  image  processing  techniques  can 
improve  the  amount  of  information  extracted  from  the  photos.  The  goal  of  this  project  was  to 
demonstrate  how  DoD  and  other  organizations  can  make  more  effective  use  of  historical  aerial 
photography  for  environmental  cleanup  support. 

2.1  BACKGROUND 

TerraSpectra  Geomatics  recently  provided  support  to  the  Albuquerque  District  of  the  U.S.  Anny 
Corps  of  Engineers  (USAGE)  to  develop  a  statewide  Geographic  Information  System  (GIS) 
database  for  New  Mexico  FUDS.  The  project  had  a  special  emphasis  on  MMRP  sites.  As 
property  and  range  boundaries  were  initially  developed  from  existing  US  ACE  ASR  documents, 
potential  limitations  of  the  ASR  historical  photo  reviews  were  noted.  Color  digital  orthophotos 
(2005)  were  used  to  verify  the  locations  of  MMRP  site  features.  Significant  offsets  between  the 
ASR  range  locations  and  their  orthophoto  locations  became  evident  during  quality  control  (QC) 
checks  of  the  data.  Although  ranges  are  broadly  defined  by  large  buffers  around  targets,  detailed 
Sis  that  rely  on  the  original  ASR  locations  could  provide  erroneous  results. 

Possible  causes  for  the  locational  offsets  and  missed  features  include: 

•  Orthophotos  were  not  widely  available  when  much  of  the  ASR  work  was 
conducted. 

•  Photographic  prints  were  used  rather  than  positive  transparencies  (diapositives). 

•  Pis  were  performed  using  pocket  stereoscopes. 

•  Photo  dates  and/or  scales  used  were  not  sufficient  for  identifying  range  features. 

Organizations  routinely  conducting  historical  photo  analyses  for  environmental  forensics1 
usually  make  use  of  diapositives  (film  positive  transparencies)  whenever  they  are  available. 
Diapositives  have  superior  resolution  and  dynamic  range  compared  to  prints  and  are,  therefore, 
preferred  for  direct  viewing  or  scanning.  The  quality  of  viewing  and  scanning  equipment  can 
also  affect  interpretation  results,  as  can  the  expertise  level  of  the  photo  interpreters. 

Since  the  1930s,  there  have  been  several  national  programs  to  collect  aerial  photography 
throughout  the  United  States.  This  photography  has  special  utility  for  large  area  studies.  The 
demonstration  of  techniques  for  Wide-Area  Assessment  (WAA)  was  a  recent  focus  area  of  the 
DoD  Environmental  Security  Technology  Certification  Program  (ESTCP)  (ESTCP,  2006).  A 
goal  of  this  program  was  to  demonstrate  cost-effective  methods  to  allocate  cleanup  resources. 


i 


Such  as  the  U.S.  Environmental  Protection  Agency’s  (USEPA)  Environmental  Photo  Interpretation  Center. 
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These  efforts  were  based  on  a  multistage  or  layered  approach  (high  airborne,  helicopter,  and 
ground)  using  a  suite  of  sensors.  These  sensor  systems  included  orthophotography,  light 
detection  and  ranging,  synthetic  aperture  radar,  hyperspectral  imaging,  magnetometer  arrays,  and 
electromagnetic  induction  arrays.  One  limitation  noted  in  studies  using  optical-based  techniques 
was  that  the  passage  of  time  may  result  in  surface  features  becoming  obscure.  The  more 
extensive  use  of  historical  photography  can  provide  a  baseline  WAA  methodology  that  can  help 
minimize  this  limitation.  This  demonstration  project  addresses  several  aspects  of  how  DoD  and 
other  organizations  could  make  better  use  of  historical  photography. 

2.2  OBJECTIVE  OF  THE  DEMONSTRATION 

The  primary  objective  of  this  project  was  to  compare  and  evaluate  results  from  three  methods  for 
the  extraction  of  information  from  historical  photography.  The  three  methods  compared  were: 

1 .  Existing  ASR  mapping  procedures — used  as  a  baseline 

2.  Best  standard  practices — based  on  different  film  media  and  interpretation 
equipment 

3.  Advanced  DIP — with  image  restoration  and  enhancements. 

There  was  a  specific  focus  on  the  visual  identification  and  mapping  of  WWII-era  practice  and 
DBRs  and  their  related  target  features.  The  improved  techniques  should  also  prove  useful  for 
other  types  of  environmental  cleanup  applications  where  the  identification  and  mapping  of 
historical  features  could  better  focus  efforts. 

Eight  project  study  sites  were  selected.  Sites  ranged  from  a  1 -square  mile  site  with  a  single 
target  to  a  site  of  about  774  square  miles  with  10  ranges.  This  provided  a  reasonable  sample  size 
in  tenns  of  the  number  of  ranges  (29)  and  target  features  (79).  The  sites  were  not  randomly 
selected.  They  included  four  properties  with  several  well-defined  ranges  and  four  properties  for 
which  range  locations  had  not  been  established  in  the  New  Mexico  FUDS  database.2 

A  secondary  objective  of  this  project  was  to  evaluate  potential  improvements  in  the  detection  and 
mapping  of  high  explosive  (HE)  bomb  craters.  If  successful,  this  would  have  provided  an 
improved  means  to  distinguish  between  practice  and  DBR  locations.  This  was  approached  using 
both  visual  interpretation  and  terrain  modeling  methods.  The  high  resolution  terrain  modeling 
approach  was  recognized  as  an  exploratory  study  element.  The  results  for  both  approaches, 
discussed  in  more  detail  later,  did  not  demonstrate  any  improved  performance  over  existing 
procedures. 

A  final  objective  of  the  demonstration  was  to  provide  for  technology  transfer.  The  development 
of  general  guidelines  for  the  processing  and  interpretation  of  historical  photography,  with  an 
emphasis  on  FUDS  applications,  supports  this  objective.  This  report  and  subsequent  technical 
presentations  and  publications  will  be  used  to  more  broadly  disseminate  the  results. 


2  During  the  course  of  this  project,  additional  Preliminary  Assessment  (PA)  studies  provided  accurate  locations  for 
two  of  the  four  unmapped  ranges 
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2.3  REGULATORY  DRIVERS 


The  environmental  cleanup  of  fonner  DoD  ranges  is  generally  conducted  under  authority  of  the 
Comprehensive  Environmental  Response,  Compensation,  and  Liability  Act.  Detailed  ground 
investigations  and  cleanup  would  be  prohibitively  expensive  unless  well-focused  to  those  areas 
most  likely  to  contain  unexploded  ordnance  or  munitions  contaminants.  The  use  of  historical 
photography  can  provide  a  unique  temporal  element  for  large  area  assessments  and  facilitate  the 
prioritization  of  other  survey  technologies.  The  use  of  aerial  photography  is  widely  understood 
and  well  accepted  by  the  regulatory  community  (ITRC,  2003). 
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3.0  TECHNOLOGY 


Key  technologies  involved  in  this  project  included  the  application  of  PI,  DIP,  photogrammetry, 
and  GIS  database  development.  Image  restoration,  specifically  blur  removal,  was  systematically 
applied  prior  to  standard  image  enhancements  (e.g.,  brightness  and  contrast  adjustments  and 
image  sharpening)  as  images  were  preprocessed  for  digital  stereo  viewing  and  orthophoto 
generation. 

3.1  TECHNOLOGY  DESCRIPTION 

Much  of  the  original  ASR  mapping  was  done  by  visually  matching  historical  photographic 
features  to  topographic  maps.  This  method  is  dependent  upon  distinct  landscape  features  and 
can  sometimes  prove  difficult  and  lead  to  large  positional  errors.  A  wealth  of  historical 
documentation  and  initial  site  reviews  were  developed  in  support  of  the  ASR  program,  but  the 
results  must  be  carefully  reviewed  before  proceeding  with  more  expensive  site-specific  field 
activities.  It  is  important  to  recognize  the  potential  limitations  of  the  earlier  ASR  work  as 
follow-on  activities  are  undertaken. 

3.1.1  Image  Restoration  and  Enhancement 

Until  the  1960s  for  military  systems,  and  much  later  for  most  commercial  systems,  aerial 
cameras  lacked  forward -motion  compensation.  This  is  a  precise  mechanical  adjustment  that  is 
made  to  account  for  aircraft  movement  during  film  exposure  (McDonnell  Douglas,  1983).  Lens 
quality  has  also  been  improved  in  newer  camera  systems.  Lens  quality  and  motion  blur  are  key 
elements  affecting  image  resolution  and  overall  image  quality  (ASPRS,  1980).  Modern  digital 
photogrammetry  systems  rely  on  the  high  quality  optics  and  automated  forward-motion 
compensation  available  with  newer  cameras.  Image  restoration  procedures  are  generally  not 
needed  nor  available  in  these  digital  or  “softcopy”  systems.  The  application  of  image  restoration 
techniques,  however,  can  potentially  improve  the  quality  of  historical  aerial  photos  and  provide 
the  basis  for  improved  interpretations.  Image  restoration  is  a  well-studied  field  that  is  distinctly 
different  from  image  enhancement.  In  certain  circumstances  where  lens  blur  or  motion  can  be 
well-characterized,  image  restoration  can  reveal  infonnation  that  cannot  be  obtained  using 
routine  image  enhancement  techniques  (SWGIT,  2002;  Ben-Ezra,  2004;  and  Simoncelli,  2005). 
The  motion  blur  example  shown  in  Figure  1  demonstrates  this. 
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Figure  1.  Example  of  image  restoration  for  motion  blur  correction. 

Original  image  (left)  and  motion  blur  corrected  image  (center)  of  aerial  imagery  of  an  orchard.  Image  enhancement 
techniques  cannot  recover  the  imagery  detail  apparent  in  the  restored  image.  The  amount  of  motion  blur  in  this 
image  is  severe  and  not  typical  of  historical  aerial  photography. 

Image  enhancement  techniques  are  now  widely  available  for  digital  imagery.  Common 
functions  are  brightness  and  contrast  adjustments,  and  edge  enhancement  or  sharpening.  It  was 
noted  during  the  course  of  this  project  that  near  real-time  adjustments  were  often  important  to 
optimize  the  resulting  image  quality.  Localized  adjustments  were  often  required  to  enhance 
specific  features  of  interest.  ERDAS®  Imagine  and  Adobe  ®  Photoshop  were  used  for  most  of  the 
image  processing.  Figure  2  provides  an  example  of  image  enhancements  applied  to  historical 
photography  of  a  practice  bombing  range  located  in  New  Mexico. 

3.1.2  Photogrammetry  Using  Historical  Photos 

Another  area  addressed  during  this  project  was  the  lack  of  camera  calibration  reports  for  some 
sets  of  older  historical  aerial  photography.  Digital  photogrammetry  systems  rely  on  calibration 
reports  to  correct  for  some  lens  distortions.  The  lack  of  this  information  requires  the  use  of  non¬ 
standard  procedures  for  generating  digital  orthophotos.  While  simple  registration  techniques  are 
widely  applied,  a  very  limited  number  of  examples  of  historical  orthophotos  have  been  reported 
in  the  literature  (Slonecker,  2009).  For  this  project,  the  ERDAS  Leica®  and  Mira  Solutions® 
photogrammetry  suites  were  used  for  the  photogrammetric  processing.  The  ERDAS  suite  now 
includes  generic  camera  models  and  procedures  for  use  when  calibration  reports  are  not 
available. 
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Figure  2.  Example  of  digital  enhancement  and  enlargement  of  1951  scanned  photograph. 

The  original  scale  of  the  9x9-inch  photograph  was  1:28,400.  It  is  shown  here  at  approximately  1:40,000  scale  (70% 
of  the  original).  The  yellow  box  area  is  the  location  of  the  practice  bombing  range  that  is  shown  enlarged  in  the 
inset.  The  enlargement  and  enhancement  of  the  range  area  shows  a  distinctive  rectangle  outline  encompassing  a 
ship  target.  The  outer  target  circle  diameter  measures  1000  ft  and  the  rectangle  measures  800  ft  in  length.  The 
photo  interpretation  of  prints  for  the  USACE  2005  Preliminary  Assessment  of  this  New  Mexico  site  (K06NM0449) 

identified  the  circular  target  but  missed  the  ship  target. 

3.1.3  Geographic  Information  Systems 

The  use  of  a  GIS  framework  for  organizing  site  imagery  and  other  data  can  significantly  improve 
interpretation  and  analysis  results.  Geospatially  organized  collateral  data  and  information  can 
often  assist  interpretation  and  analyses.  The  Environmental  Systems  Research  Institute  (ESRI)® 
ArcGIS®  software  was  used  to  provide  this  functionality  and  organize  interpretation  results. 

3.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

A  primary  advantage  of  historical  photography  is  that  it  can  provide  the  appropriate  temporal 
element  for  past  activities  of  interest.  Aerial  photographs  are  unique  in  providing  a  visual  record 
or  “snapshot  in  time”  of  historical  site  conditions.  When  suitable  photography  is  available  it  can 
prove  extremely  cost-effective,  since  no  mission  deployment  costs  are  involved.  Only  archive 
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data  searches  and  reproduction  or  digital  scanning  costs  are  involved  to  acquire  imagery.  See  an 
example  in  Figure  3. 

A  primary  limitation  of  aerial  photography  and  other  optical-based  techniques  is  that  they  are  not 
directly  sensing  the  presence  of  munitions  but  must  make  use  of  a  surrogate  or  indicator.  Aerial 
photography  requires  a  surface  expression,  such  as  visually  distinct  range  target  features  or  bomb 
craters,  as  an  indicator  for  the  likely  presence  of  munitions-related  materials.  The  use  of 
historical  photography  is  also  limited  by  the  type  and  quality  of  photography  available. 
Obviously  there  are  no  retroactive  options  to  change  the  original  acquisition  parameters. 

It  was  noted  during  this  study  that  photographic  archives  are  continuing  to  expand  as  historical 
photos  are  identified  and  added  to  the  collections.  With  new  finding  aids,  especially  those  based 
on  geographic  coordinate  based  databases,  they  are  also  becoming  easier  to  search.  One  result  of 
this  is  that  prior  search  results  cannot  be  relied  upon  as  complete;  new  searches  often  identify 
additional  sets  of  historical  photos.  Prior  ASR  photo  searches  generally  made  use  of  the  primary 
government  archives  of  historical  photos:  the  National  Archives  and  Records  Administration 
(NARA),  the  U.S.  Geological  Survey  EROS  Data  Center  (USGS-EDC),  and  the  U.S. 
Department  of  Agriculture  Aerial  Photography  Field  Office  (APFO).  Additional  photography  is 
often  available  from  other  federal  organizations  and  especially  state  and  local  resources.  The 
region  specific  resources  are  more  numerous  and  sometimes  more  difficult  to  readily  identify. 
Primary  regional  sources  include  private  aerial  photo  firms,  university  libraries,  and  state  and 
local  governmental  organizations.  Transportation  departments  are  often  a  good  source  at  the 
state  and  local  level.  The  validation  effort  for  this  project  made  effective  use  of  state  and  local 
resources  to  obtain  selected  copies  from  their  photographic  archive  holdings. 

The  shelf-life  of  film  is  limited  and  a  concern  for  all  archives.  Shelf-life  can  vary  significantly, 
from  a  few  years  to  many  decades,  depending  on  storage  conditions  such  as  temperature  and 
humidity.  Early  film  was  nitrate-based,  which  had  the  drawback  of  decomposing  and  being 
extremely  flammable.  Acetate-based  “safety  film”  was  subsequently  developed,  but  cellulose 
triacetate  degradation  was  reported  within  a  decade  of  its  introduction  in  1948.  As  it  became 
better  understood,  this  problem  has  become  known  as  the  “vinegar  syndrome.”  Once  started, 
rapid  deterioration  can  render  the  acetate  film  element  past  the  point  of  transferability  to  another 
medium.  Professional  archives  have  standard  procedures  to  monitor  and  handle  this  problem. 

Conversion  to  digital  format  is  one  option  for  long-tenn  aerial  photo  storage.  The  USGS-EDC 
no  longer  provides  film  copies  of  their  extensive  holdings  of  over  8  million  frames  of  aerial 
photography.  Only  digital  scans  of  their  film  are  now  available  as  a  standard  product.  Moving 
towards  digital  imagery  is  therefore  now  a  requirement  in  order  to  use  some  historical  archives, 
and  this  conversion  trend  is  expected  to  continue. 
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Figure  3.  Temporal  pairs  of  targets. 

These  matched  pairs  of  photographs  illustrate  former  bombing  ranges  located  on  the  Guadalupe  Bombing  and 
Gunnery  Range  (K06NM033),  located  in  New  Mexico.  The  top  pair  illustrates  a  demolition  range  (designated  D-6) 
that  remains  distinct  on  the  2005  imagery,  although  the  craters  evident  in  1950  (frame  55)  are  no  longer  apparent. 
The  bottom  pair  is  a  practice  bombing  range  (PB-1)  that  is  composed  of  a  circular  target  adjacent  to  a  large  “Y” 
shaped  air-to-air  navigation  feature  (1950  frame  53).  This  range  is  no  longer  distinct  on  the  2005  imagery. 

The  1950  photography  was  acquired  at  an  original  scale  of  1:44,000.  The  2005  orthophotos  were  generated 
at  a  pixel  resolution  of  1  m  for  a  statewide  orthophoto  mapping  program.  The  images  above  are  presented 

at  a  nominal  scale  of  1:20,000. 


11 


This  page  left  blank  intentionally. 


12 


4.0  PERFORMANCE  OBJECTIVES 

Performance  objectives  for  this  demonstration  project  are  summarized  in  Table  1. 

Table  1.  Performance  objectives. 


Performance  Objective 

Metric 

Success  Criteria 

Quantitative  Performance  Objectives 

1.  Visual  identification  of 
all  bombing  range  areas 

Percent  detected 

Location  of  ranges 

100%  detection 

2.  Limited  false  alarms 
for  bombing  range  areas 

Percent  of  areas 
incorrectly  identified  as 
ranges 

Location  of  ranges 

<  50%  false  identifications 

3.  Identify  all  range 
target  features  (outlines  of 
ships,  docks,  airstrips,  etc.) 

Percent  detected 

Location  of  range 
targets 

>  90% 

4.  Limited  false  alarm 
rate  for  range  target 
features  (above) 

Percent  of  features 
incorrectly  identified  as 
range  target  features 

Location  of  range 
targets 

<  50%  false  identifications 

5.  Identification  of  ranges 
with  craters  (inferred  use 
of  HE  munitions) 

Percent  detected 

Location  of  craters 
within  or  near  ranges 

>  75%  detection 

6.  Feature  mapping 
location  accuracy 

Average  range  and  target 
feature  mapping  location 
error 

Feature  centroid 
location  mapped  from 
historical  photos  onto 
their  corresponding 
orthophoto  locations 

<  10  m  distance  offset 

7.  Interpretation 
production  rate 

Time  required  to  analyze 
each  stereo-pair  of  photos 

Log  of  analysis  time 
accurate  to  10  minutes 

Analysis  time:  <  1  hour  per 
photo  pair 

8.  Orthophotos  from 
historical  photos 

Average  tie  and  check 
point  location  offset 
errors 

Location  accuracy  of 
distinct  features 
compared  to  USGS  (or 
similar)  orthophoto 

<  10  m  distance  offset  for  tie 
points 

<  15  m  offset  for  distinct 
check  point  features 

9.  DEM  from  historical 
photos 

Percent  correct  detection 
of  craters 

Demolition  bomb  crater 
feature  locations 

Improved  crater  detection 
performance  using  restored 
imagery  versus  non-re  stored 
imagery 

Qualitative  Performance  Objectives 

10.  Ease  of  use  and 
technology  transfer 

(Not  applicable) 

Feedback  from  analysts 
on  the  usability  of  the 
different  procedures 
and  products  developed 

Completion  of  general 
guidelines  or  protocols  for 
use  of  historical  photos  and 
professional  publication(s) 
of  results 

DEM  =  digital  elevation  model 
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5.0  SITE  DESCRIPTIONS 


The  emphasis  of  this  technology  demonstration  project  was  on  large  practice  and  DBRs.  These 
ranges  were  originally  developed  and  operational  during  WWII  and  shortly  thereafter  supported 
bombardier  crew  training.  The  target  features  constructed  on  these  ranges  were  generally  large 
and  distinct,  as  the  training  missions  were  based  on  visual  target  identifications. 

5.1  SITE  SELECTIONS 

Eight  FUDS  MMRP  sites  were  selected  for  the  demonstration.  Six  of  the  sites  were  located  in 
New  Mexico,  and  two  were  in  Texas.  Figure  4  presents  a  map  of  the  study  site  locations.  These 
sites  were  believed  to  be  generally  representative  of  environmental  and  bombing  range 
conditions  and  target  features  present  in  the  Southwest  region  of  the  United  States.  Several 
criteria  were  used  in  the  selection  of  the  test  sites,  including  range  size,  availability  of  WAA  data 
for  validation,  and  sites  where  target  features  had  not  been  previously  photo  verified  or 
unexpected  HE  debris  had  been  encountered  during  field  investigations  (see  Table  2).  More 
detailed  site  histories,  characteristics  and  selection  rationale  for  the  eight  FUDS  MMRP  sites  that 
were  used  for  this  study  are  provided  in  the  project  final  report  (TerraSpectra,  2010). 
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Table  2.  Project  study  sites 


Project  #  - 
FUDS  ID 
Study  Date 

RAC(1) 

Score 

Site 

Name 

Active 

Period 

Range 

Acres 

Property 

Acres 

Comments 

NEW  MEXICO  SITES 

#1  -  K06NM0333 
ASR 2001 

4 

Guadalupe  Bombing  and  Gunnery 
Range 

1943-1956 

12,539 

495,053 

Combination  of  several  practice  and  demolition 
ranges,  with  several  unmapped  navigation 
markers 

#2  -  K06NM0394 
ASR 1997 

4 

Deming  Air  Force  Base  (AFB) 
Precision  Bombing  Range  (PBR) 
#10 

1943-1947 

649 

960 

Practice  bombing  range;  no  features  evident  in 
photos  used  for  original  ASR;  range  considered 
“missing”  (2) 

#3  -  K06NM0445 
ASR  1994 

2 

Kirtland  AFB  Ranges  -  West  Mesa 

1941-1945 

1298 

15,246 

Combination  of  multiple  practice  and  one 
demolition  range;  WAA  study  site 

#4  -  K06NM0449 

PA  2005 

4 

Kirtland  AFB  PBR  #S-12 

1942-1946 

649 

640 

Considered  “missing”  practice  range  at  project 
start;  range  found  off-site  of  original  property 
that  was  field  inspected 

#5  -  K06NM0499 
ASR  1998 

4 

Walker  AFB  DBR#35 

1944-1945 

649 

1000 

Considered  “missing”  demolition  range;  range 
target  features  were  not  evident  in  the  ASR 
photo  set 

#6  -  K06NM0619 

PA  2004 

4 

Kirtland  AFB  PBR  #18  Target  S-5 

1943-1946 

649 

640 

Considered  “missing”  practice  range  at  project 
start;  range  found  off-site  of  original  property 
that  was  field  inspected 

TEXAS  SITES 

#7  -  K06TX0186 
ASR  2000 

3 

Midland  AAF  Target  Range  #14 

1942-1947 

1646 

1646 

Unexpected  FIE  debris  found  during  SI;  several 
practice  range  target  features  and  crater  area 

#8  -  K06TX0267 

SI  2009 

4 

Dalhart  PBRs  #3  and  #4 

1943-1945 

16,581 

16,581 

Combination  of  two  practice  and  one 
demolition  range,  with  several  target  features 
present;  unmapped  HE  range 

(1)  RAC:  Risk  Assessment  Code  assigned  by  USACE  for  each  site  during  initial  assessments  (l=highest  risk,  5=lowest  risk). 


<2)  When  this  ESTCP  project  was  proposed,  four  New  Mexico  FUDS-MMRP  sites  were  not  mapped  via  PI  in  the  available  FUDS  related  documents;  all  four  were  included  as  project  sites  with  a 
secondary  project  goal  of  locating  the  “missing”  ranges;  PA  documents  for  two  of  the  sites  (K06NM0449  and  K06NM0619)  were  subsequently  made  available  that  correctly  locate  the  range 
locations  (but  miss  the  battleship  target  on  K06NM0449).  These  more  recent  range  maps  were  used  for  the  comparative  analyses. 

(3)  The  original  1998  ASR  did  not  examine  aerial  photos,  so  individual  ranges  were  not  mapped  and  the  entire  property  area  was  considered  a  range;  results  from  the  2009  SI  were  used  for  the 
comparative  analyses. 


6.0  TEST  DESIGN 


Existing  range  maps  developed  in  support  of  prior  ASRs  (or  the  selected  PA  and  SI  studies  noted 
in  Table  2)  were  used  as  the  baseline  source  of  range  feature  infonnation.  These  maps  are 
available  online  from  the  MMRP  SI  section  of  the  2008  Defense  Environmental  Restoration 
Program  (DERP)  Annual  Report  to  Congress  (DERP,  2008).  It  was  anticipated  that  some  “new” 
locations  for  unmapped  bombing  range  features  might  be  identified.  Most  of  these  features  were 
expected  to  be  self-evident  upon  visual  inspection.  Only  a  few  questionable  features  were 
expected  to  require  field  validation. 

6.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

The  basic  experimental  design  for  this  demonstration  was  to  compare  interpretation  results  from 
existing  range  maps  (ASR  or  subsequent  studies)  to  those  obtained  using  “best  practice”  PI 
procedures  and  digital  image  analysis  procedures.  The  new  Pis  made  use  of  standard  procedures 
developed  by  the  USEPA  and  other  organizations  routinely  involved  in  environmental  forensics. 
An  experienced  analyst  interpreted  film  diapositives  using  stereo  zoom  viewing  equipment.  A 
separate  review  of  the  results  by  another  senior  analyst  was  used  to  provide  a  QC  check.  All 
differences  in  interpretations  were  resolved  by  a  visual  review  with  discussion  and  consensus. 
The  potential  influence  of  differences  among  different  professional  interpreters  and  the  amount 
of  time  available  to  conduct  the  analyses  was  recognized  but  was  not  addressed  as  part  of  this 
study.  All  interpreters  had  at  least  10  years  professional  experience. 

The  digital  processing  alternative  involved  similar  interpretation  procedures,  but  was  based  on 
digitally  pre-processed  imagery  rather  than  film  products.  There  are  many  potential  photo 
scanning  resolutions,  image  enhancement  and  restoration  algorithms,  and  parameter  settings 
possible  for  the  digital  processing.  It  was  not  practical  to  quantitatively  test  the  numerous 
permutations  possible.  As  such,  an  experienced  analyst  selected  an  appropriate  set  of  algorithms 
and  settings  to  apply.  Additional  interactive  enhancements  were  applied  “on  the  fly”  by  the 
analysts  to  assist  any  specific  feature  interpretations  (e.g.,  a  locally  different  contrast)  that  was 
required  to  best  interpret  a  specific  feature.  As  with  the  film-based  alternative,  the  initial 
analyst’s  results  were  reviewed  by  another  senior  analyst  and  any  differences  in  interpretations 
resolved  by  review  with  discussion  and  consensus  for  final  interpretations. 

To  avoid  possible  crossover  influence  among  the  analyses,  different  pairs  of  analysts  were 
involved  for  the  PI  and  digital  image  analyses.  The  analysts  were  not  provided  with  any  site 
specific  identification  or  descriptive  infonnation.  The  historical  imagery  was  provided  without 
any  site  location  or  descriptions  other  than  film  date  and  scale.  Limiting  access  to  collateral 
infonnation  is  nonnally  not  recommended,  as  it  can  hinder  interpretations.  This  extra  limitation 
was  applied  to  avoid  possible  influences  from  the  ASR  results  to  the  new  interpretations. 

In  addition  to  the  primary  objectives  of  range  and  feature  identification  and  mapping,  the 
locational  accuracy  of  orthophotos  developed  from  the  historical  photography  was  tested  using 
USGS  and  similar  quality  orthophotos.  Recent  orthophotos  were  assumed  geometrically  correct 
and  used  as  the  basis  for  control  of  the  historical  orthophotos  that  were  developed.  Location 
accuracy  was  estimated  by  evaluating  offsets  between  matched  pairs  of  visually  distinct  feature 
locations  in  the  orthophotos. 
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6.2  EQUIPMENT  SPECIFICATIONS 


The  specifications  for  historical  aerial  photography  are  inherently  predefined  and  not  subject  to 
change.  Available  metadata  infonnation  is  often  limited  to  statements  of  film  type  (e.g.,  black 
and  white  versus  color),  scale,  and  area  of  coverage.  A  range  of  film  scanning  resolutions  was 
possible  for  the  digital  analyses.  Photogrammetric  applications  generally  require  higher 
resolutions  and  geometric  fidelity  compared  to  many  other  image  processing  applications. 
Figure  5  provides  an  example  of  the  four  different  standard  scan  resolutions  used  by  USGS- 
EDC. 


Figure  5.  Comparison  of  film  scanning  resolutions. 

Circular  target  range  on  Site  #8  -  K06TX0267  (Dalhart)  1954  photograph  (frame  3994).  The  bottom  three  images 
were  from  a  high  quality  photogrammetric  scanner,  while  the  top  image  is  from  a  graphics  scanner  used  to  generate 
browse  images.  All  four  image  scans  are  from  the  USGS-EDC.  The  target  circle  appears  slightly  degraded  on  the 
25  micron  scan  and  would  not  be  identified  on  the  40  micron  scan  without  prior  knowledge  of  its  location.  The 
original  photograph  was  acquired  in  1954  at  a  film  scale  of  1:60,000. 

The  outer  target  circle  is  1000  ft  in  diameter. 
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The  USGS-EDC  initiated  a  program  in  2009  to  systematically  scan  their  aerial  photo  archive 
collections  to  a  standard  fonnat  of  25  microns.  They  have  also  discontinued  the  production  of 
fdm  or  print  hardcopies  for  aerial  photo  products.  USGS-EDC  photo  products  are  now  delivered 
in  digital  fonnat  only. 

6.3  DATA  COLLECTION 

Historical  photography  was  acquired  to  match  the  corresponding  set  of  photography  acquired  for 
the  ASR  studies.  Table  3  provides  a  summary  of  the  photo  years  and  scales  acquired  for  the 
baseline  studies.  Detailed  citations  (mission  identifiers,  frame  numbers,  etc.)  are  available  from 
the  corresponding  site  study  reports. 

Table  3.  Photo  data  sets  used  for  the  comparative  analyses. 


Site# 

Site 

Study 

Photo  Year(1) 

Photo  Scale 

1 

Guadalupe  Bombing  &  Gunnery  Range 
K06NM0333 

ASR  2001 

1950 

1:44,000  ' 

1972-74 

1:32,000  ! 

1996-98 

1:40,000  ' 

2 

Deming  AFB  PBR#10 

K06NM0394 

ASR  1997 

1974 

1:40,000 

3 

Kirtland  AFB  PBR  -  West  Mesa 
K06NM0445 

ASR  1994 

196712' 

1:26,000  ' 

4 

Kirtland  AFB  PBR  #S-12 

K06NM0449 

PA  2005  (J) 

1951 

1:28,400  | 

1997 

1:40,000 

5 

Walker  AFB  DBR#35 

K06NM0499 

ASR  1998 

1971-72 

1:24,000  1 

6 

Kirtland  AFB  PBR  #18  Target  S-5 
K06NM0619 

PA  2004  l4) 

1946 

1:35,000 

1971 

1:40,000  ; 

1996 

1:40,000 

7 

Midland  AAF  Target  Range  #14 
K06TX0186 

ASR 2000 

1946 

1:20,000  ; 

1966 

1:20,000 

8 

Dalhart  PBR  Ranges  #3  and  #4 

K06TX0267 

SI  2009  0) 

1954 

1:60,000 

(l)  Photo  mission  campaigns  can  span  months  to  a  few  years;  the  ranges  for  three  multiyear  missions  are  shown. 


<2)  The  1994  ASR  notes  several  photo-based  observations  of  target  features  but  does  not  provide  a  detailed  photo  date  listing.  There  is  one 
reference  to  photos  taken  approximately  1971.  The  only  sitewide  photo  availability  near  this  timeframe  was  1967,  which  was  used  as  the 
basis  for  the  comparative  analysis.  It  is  possible  that  the  ASR  used  a  combination  of  other  unspecified  dates  of  photography. 

(3)  No  ASR  was  available  at  the  start  of  this  project;  the  2005  PA  was  used  as  the  source  for  comparisons. 

(4)  No  ASR  was  available  at  the  start  of  this  project;  the  2004  PA  was  used  as  the  source  for  comparisons. 

<:>)  The  ASR  (1998)  for  this  site  did  not  examine  any  photography;  the  2009  SI  map  was  used  for  comparisons. 


Standard  overlapping  sets  of  photos  were  acquired  to  allow  stereo  viewing.  Stereo  viewing 
proved  less  important  than  anticipated,  as  nearly  all  range  target  features  were  located  on  flat 
terrain.  Stereo  coverage  did  prove  useful  to  eliminate  the  possible  misinterpretations  of  several 
film  artifacts,  which  are  more  common  in  some  of  the  older  sets  of  photography.  Figure  6 
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provides  an  example  of  a  circular  film  processing  artifact  that  might  be  misinterpreted  as  a 
bombing  range  target  feature  if  only  the  one  frame  of  photography  was  examined. 
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Figure  6.  Stereo  pair  of  1950  photographs  with  artifact. 

The  500-ft  circular  feature  on  the  photo  (enlarged  from  frame  58)  is  a  film  processing  artifact.  It  could  be 
misinterpreted  as  a  target  feature  based  on  its  circular  shape,  size,  and  intersection  with  a  road  that  could  have 
provided  range  access.  The  feature  is  missing  in  the  overlapping  photo  (frame  57),  which  proves  that  the  feature  is  a 
film  processing  artifact.  The  photos  are  from  Site  #1  -  K06NM0333  (Guadalupe). 


Photos  were  acquired  in  both  diapositive  (film  transparency)  and  scanned  digital  format.  The 
USGS-EDC  has  transitioned  to  a  digital-only  distribution  policy  for  copies  of  aircraft  and 
satellite  imagery  from  their  archives.  This  makes  the  future  capability  to  handle  digital  imagery 
mandatory  for  some  photography.  Vendors  supporting  NARA  currently  support  both  film 
duplicate  and  digital  scans  of  the  archive  holdings.  These  and  other  sources  were  used  to  acquire 
additional  photography  that  was  used  for  validations. 

6.4  VALIDATION 


Possible  interpretation  errors  can  be  categorized  into  two  classes  -  errors  of  omission  (Type  I) 
and  errors  of  commission  (Type  II).  In  the  context  of  this  project,  an  error  of  omission  would  be 
to  miss  the  detection  of  a  range  feature.  An  error  of  commission  would  be  to  misidentify  a  non¬ 
range  feature  by  calling  it  a  range  feature.  Although  both  interpretations  would  be  considered 
errors,  for  this  application  an  omission  error  would  be  considered  more  significant  than  a 
commission  error  -  i.e.,  missing  a  range  would  be  more  significant  than  calling  a  non-range 
feature  a  range  feature.  Similarly,  missing  sites  where  HE  bombs  were  used  would  be 
considered  more  significant  than  missing  sites  used  for  only  practice  bombs. 
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Significant  effort  was  extended  to  validate  all  features  to  the  maximum  extent  that  cost 
considerations  allowed.  Validation  efforts  used  a  variety  of  sources  and  methods  and  a 
“convergence  of  evidence”  approach  to  determine  if  a  feature  was  considered  validated.  The 
primary  and  most  productive  method  was  to  conduct  updated  and  more  extensive  searches  of 
historical  photo  archives  and  then  acquire  better  dates  or  scales  of  photography. 

Marginal  dates  of  photography  and  scale  concerns  were  noted  for  the  following  sites: 

Date  Concerns  Scale  Concerns 

Site  #2  (Deming)  -  1974  Site  #1  (Guadalupe)  -  1:44,000 

Site  #3  (Kirtland)  -  1967  Site  #2  (Deming)  -  1:40,000 

Site  #5  (Walker)  -  1971/1972  Site  #8  (Dalhart)  -  1:60,000 

More  suitable  dates  and  scales  of  photography  were  identified  and  selected  photographs  were 
acquired,  in  digital  format,  for  all  five  of  these  sites.  Selected  sets  of  photographs  taken  prior  to 
range  developments  were  also  acquired  to  assist  the  interpretation  of  specific  features  of  interest 
(see  Table  4).  Scanning  options  available  from  some  of  the  additional  sources  were  highly 
variable  in  terms  of  scanning  resolution  and  quality  of  equipment.  In  some  instances,  validation 
photos  were  only  available  as  digital  scans  of  photographic  prints  using  graphic  scanners. 
Although  significantly  less  optimal  in  terms  of  image  quality,  the  specific  dates  and  scales 
available  were  generally  considered  to  be  more  important  for  the  validations  than  the  media  or 
type  of  scanner. 


Table  4.  Additional  photography  acquired  to  assist  range  and  feature  validations. 


Site# 

Site 

Photo  Year 

Photo  Scale 

1 

Guadalupe  Bombing  &  Gunnery  Range 

K06NM0333 

1943 

1:  50,000 

1946 

1:  24,000 

1948 

1:  27,230 

1958 

1:  17,200 

2 

Deming  AFB  PBR#10 

K06NM0394 

1942* 

1:  56,000 

1951 

1:  20,000 

1953 

1:  54,000 

1956 

1:  31,680 

3 

Kirtland  AFB  PBR  -  West  Mesa 

K06NM0445 

1935* 

1:  44,000 

1945 

1:  21,400 

1951 

1:  24,000 

5 

Walker  AFB  DBR#35 

K06NM0499 

1946 

1:  31,680 

1954 

1:  63,000 

8 

Dalhart  PBR  Ranges  #3  and  #4 

K06TX0267 

1941* 

1:  20,000 

1953 

1:  20,000 

*Photo  dates  prior  to  range  operational  periods. 


The  additional  photography  provided  sufficient  source  material  to  validate  nearly  all  of  the  photo 
interpreted  features.  It  also  identified  several  features  that  had  been  missed  on  later  dates  and/or 
smaller  scales  of  photography.  After  review  of  the  additional  photography,  validation  problems 
did  persist  for  two  sites:  Site  #2  (Deming),  which  were  not  fully  resolved,  and  Site  #8  (Dalhart), 
which  were  validated  by  field  checks. 
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7.0  DATA  ANALYSIS  AND  PRODUCTS 


The  specific  processing  and  analysis  steps  that  were  used  for  the  comparison  of  baseline  and 
alternative  methods  are  discussed  in  the  following  sections. 

7.1  PHOTO  ACQUISITIONS  AND  PREPROCESSING 

Both  alternative  methods  of  data  analysis  began  with  the  acquisition  of  historical  aerial  film 
diapositives  (positive  transparencies)  for  the  study  sites.  For  direct  PI  there  was  no 
preprocessing  of  the  imagery  required.  Standard  film  quality  checks  were  used  to  document  the 
aerial  film  characteristics  (correct  coverage,  general  brightness  and  contrast,  presence  of  haze, 
clouds,  or  cloud  shadows,  general  sharpness,  any  film  processing  streaks,  etc.). 

For  digital  processing,  the  photographs  were  scanned  using  photo grammetric  quality  scanners  at 
high  resolution.  Initial  scanning  was  performed  at  the  highest  available  resolution  of  7  microns 
(3629  pixels  per  inch).  The  results  of  the  highest  resolution  scans  presented  at  full  pixel 
resolution  were  considered  too  noisy,  even  after  substantial  image  processing.  An  empirical 
comparison  of  different  scanning  results  led  to  the  selection  of  14  microns  (1814  pixels  per  inch) 
as  providing  a  more  practical  resolution  for  scanning  the  historical  photography.  The  film 
resolving  power  of  more  recent  photography  may  benefit  from  the  higher  resolution  scanning.  A 
detailed  assessment  of  optimal  scanning  resolutions  for  different  films  and  flight  acquisition 
parameters  was  beyond  the  scope  of  this  project. 

Selective  image  enhancements  and  restoration  procedures  were  applied  to  the  imagery  to  develop 
the  comparative  products.  These  included  routine  procedures  such  as  brightness/contrast  and 
edge  enhancement,  as  well  as  the  more  advanced  blur  correction,  and  photogrammetric 
procedures  for  orthophoto  and  DEM  generation. 

7.2  TRAINING  KEYS 

Image  training  keys  that  provided  examples  of  range  features  present  at  other  bombing  ranges 
(sites  not  involved  in  this  study)  were  used  to  familiarize  the  analysts  with  the  types  of  features 
to  be  identified.  A  classification  scheme  was  developed  for  the  basic  types  of  recognized  range 
features  (e.g.,  target  circles,  cross-hairs,  HE  bomb  craters,  outlines  of  ships,  docks,  fuel  storage 
tanks,  airfield,  train,  etc.).  An  “area  of  interest”  (AOI)  feature  type  was  included  to  identify 
other  features  that  might  have  been  range  related  but  could  not  be  reasonably  defined  as  range 
target  features. 

7.3  TARGET  DETECTION  AND  IDENTIFICATION 

The  two  sets  of  imagery  (film  and  digital)  were  interpreted  by  two  sets  of  experienced  image 
analysts  for  the  visual  detection  and  identification  of  features  comprising  a  bombing  range. 
Stereo-pairs  of  imagery  were  examined  for  both  types  of  interpretations.  The  analysts  recorded 
the  type  and  location  of  each  feature  and  included  a  basic  confidence  factor  for  each  feature 
mapped:  confident,  probable,  or  possible.  Additional  AOIs,  such  as  towers  and  building 
structures,  were  also  annotated  but  not  incorporated  into  the  comparative  analyses.  Film  analysis 
results  were  initially  annotated  onto  photo  overlays  and  then  transferred  into  GIS  fonnat  using 
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orthophoto  bases.  Digital  interpretations  were  directly  mapped  onto  orthophotos  using  GIS 
techniques  to  record  the  location  and  attributes  of  range  features. 

7.4  DATA  PRODUCTS 

Comparative  data  sets  of  photography  for  the  PI  and  digital  scans  for  the  DIP  analyses  were 
prepared  for  each  site.  Additional  photography  and  other  collateral  data  were  collected  to  assist 
the  validation  effort.  Figure  7  provides  a  time  series  example  of  four  dates  of  photos  for  the  N-l 
Range  area  of  Site  #3  (Kirtland  -  West  Mesa).  It  was  noted  that  in  6  years  (from  1945  to  1951) 
the  range  features  became  much  less  distinct  and  a  bridge  or  convoy  target  observed  on  the  1945 
photo  was  no  longer  apparent  by  1951.  The  much  later  1967  photography  was  used  for  the 
comparative  analysis.  By  that  time  frame,  only  the  primary  target  circle  and  cross-hairs 
remained  evident,  and  several  other  target  features  were  missed  by  the  interpreters.  The  time 
series  example  reinforces  the  critical  nature  of  acquiring  an  appropriate  set  of  historical 
photography  to  map  range  features. 

Figure  8  provides  an  example  of  a  false  alann  (commission  error)  from  Site  #3.  A  hexagon 
layout  of  building  features  is  evident,  with  several  access  roads  still  visible  on  the  1967 
photography  that  was  used  for  the  comparative  analyses.  Earlier  dates  of  photography 
documented  that  this  complex  was  developed  sometime  between  1951  and  1967,  which  is  after 
the  range  operational  period.  Both  the  PI  and  DIP  analysts  interpreted  this  feature  as  a  possible 
target. 

Site- wide  aerial  photo  coverage  acquired  in  1951  was  obtained  for  Site  #3  and  used  as  the  basis 
for  developing  digital  orthophotos.  A  mosaic  of  these  orthophotos  is  presented  as  Figure  9.  The 
historical  orthophotos  were  used  as  the  mapping  base  for  new  interpretations.  Partial  coverage 
of  the  site  was  also  obtained  for  1945,  which  was  noted  earlier.  These  additional  photos  were 
georeferenced  to  the  orthophoto  mosaic. 

Some  of  the  validation  photography  was  acquired  from  sources  that  only  had  duplicate  prints  and 
no  photogrammetric  quality  scanners  available.  Although  less  than  optimal,  in  many  instances 
this  photography  covered  critical  time  periods  and  provided  useful  infonnation  for  the 
validations.  Photos  were  also  acquired  for  three  sites  for  dates  prior  to  range  operations:  1942 
photos  of  Site  #2  (Deming),  1935  photos  of  Site  #3  (Kirtland-West  Mesa),  and  1941  photos  for 
Site  #8  (Dalhart).  These  photos  were  acquired  to  help  validate  specific  site  features  that  were 
considered  possible  pre-existing  features  that  were  not  range  activity  related.  NARA  estimates 
that  85%  of  the  continental  United  States  has  such  pre-WWII  photography  available  (NARA, 
1973). 

Validation  data  sets  included  ASR  maps  and  other  site  descriptions  available  from  various 
historical  documents.  The  site  related  maps  that  were  considered  pertinent  to  Pis  were  usually 
georeferenced.  This  facilitated  the  interpretation  process  by  allowing  the  use  of  GIS  techniques. 
Any  subsequent  analyses,  such  as  SI  results,  were  also  examined  and  were  used  whenever 
appropriate.  These  more  recent  studies  usually  included  GIS  datasets  that  could  be  readily 
incorporated  into  the  site  datasets  used  for  validations. 
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Figure  7.  Time  series  of  Site  #3  -  K06NM0445  (N-l  Range  Area). 

This  range  has  a  bridge  or  convoy  target  evident  in  the  1945  photograph  (yellow  box)  that  is  no  longer  apparent  by 
the  1951  time  frame.  The  comparative  image  analyses  in  this  study  were  based  on  the  1967  photography.  In  the 
1967  photograph  the  target  circles  and  cross-hairs  are  still  evident,  but  several  other  targets  are  no  longer  apparent. 
By  2005,  the  target  features  are  barely  discernable  or  have  been  obscured  by  development.  Historical  documents 
noted  that  the  original  cross-hairs  were  developed  using  magnetic  north  instead  of  true  north  and  had  to  be  redone 
for  correct  usage  by  navigators,  thus  explaining  the  offset  shorter  cross-hairs  that  are  most  evident  in  the  1945  photo 
Original  photo  scales  ranged  from  1:21,500  to  1:26,000.  The  nominal  presentation  scale  shown  here  is  1:10,000. 
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Figure  8.  Hexagon  building  layout  on  site  #3  -  K06NM0445  (Kirtland  -  West  Mesa). 

Small  buildings  or  sheds  around  a  central  building  complex  were  noted  on  the  1967  photos.  The  complex  was 
located  between  the  inactive  N-2  and  New  Demolition  Ranges.  These  features  were  identified  as  possible  range 
features  by  the  IP  and  DIP  analysts.  Range  activities  were  reported  to  have  ended  in  1 945  and  these  feature  are  not 
evident  in  the  1951  photographs.  Due  to  the  time  frame  of  their  development,  the  validation  effort  concluded  that 
these  were  not  related  to  the  WW11  bombing  range  activities.  The  original  photo  scale  was  1:26,000.  The 
presentation  scale  of  the  photo  shown  above  is  about  1:16,000. 
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Figure  9.  1951  digital  orthophoto  mosaic  of  Site  #3  -  K06NM0445  (Kirtland  -  West  Mesa). 

The  background  image  is  the  2005  color  orthophoto  used  for  control.  Munitions  ranges  are  outlined  and  labeled  in  yellow.  The  1951  black  and  white  orthophoto 
mosaic  was  developed  from  1 1  photos.  The  overall  FUDS  outline  is  shown  in  light  blue.  The  SORT  range  was  a  Simulated  Oil  Refinery  Target.  Distinct  HE 
craters  are  apparent  around  the  New  Demolition  target.  The  field  heavy  artillery  target  was  the  only  target  not  located  on  flat  terrain.  Careful  examination  of 
trail  networks  was  useful  at  many  sites  to  identify  likely  target  areas.  Historical  documentation  noted  the  need  for  roads  and  trails  to  service  the  targets. 
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8.0  PERFORMANCE  ASSESSMENT 


A  summary  table  of  the  validated  interpretation  results  for  the  performance  objectives  is 
presented  below  (Table  5).  More  extensive  descriptions  and  a  tabular  listing  of  site-specific 
feature  interpretation  results  are  provided  in  the  project  final  report  (TerraSpectra,  2010).  The 
success  criteria  for  three  of  the  nine  performance  objectives  were  not  fully  met:  #3,  #5,  and  #9. 

Table  5.  Performance  assessment  results. 


Performance 

Objective 

Metric 

Success  Criteria 

Results 

Quantitative  Performance  Objectives 

1.  Visual  identification 
of  all  bombing  range 
areas 

Percent  detected 

100%  detection 

ASR:  72%  (21/29)  V 

PI:  97%  (28/29)  V 

DIP:  100%  (29/29)  V 

2.  Limited  false  alarms 
for  bombing  range  areas 

Percent  of  areas 
incorrectly  identified  as 
ranges 

<  50%  false  alarms 

ASR:  0%  (0/29)  7 

PI:  3%  (1/29)  V 

DIP:  3%  (1/29)  V 

3.  Identify  all  range 
target  features  (outlines 
of  ships,  docks,  airstrips, 
etc.) 

Percent  detected 

>  90%  detection 

ASR:  42%  (33/79) 

PI:  75%  (59/79) 

DIP:  84%  (66/79) 

4.  Limited  false  alarm 
rate  for  range  target 
features  (above) 

Percent  of  features 
incorrectly  identified  as 
range  target  features 

<  50%  false  alarms 

ASR:  0%  (0/79)  ^ 

PI:  4%  (3/79)  V 

DIP:  4%  (3/79)  V 

5.  Identification  of 
ranges  with  craters 
(inferred  use  of  HE 
munitions) 

Percent  detected 

>  75%  detection 

ASR:  70%  (7/10) 

PI:  60%  (6/10) 

DIP:  70%  (7/10) 

6.  Feature  mapping 
location  accuracy 

Average  range  and  target 
feature  mapping  location 
error 

<10  meters  offset 

ASR:  28.6  m 

PI:  2.4  m  V 

DIP:  2.2  m  V 

7.  Interpretation 
production  rate 

Time  required  to  analyze 
each  stereo-pair  of  photos 

<  1  hour/photo  pair 

PI:  49  min/pair  V 

DIP:  57  min/pair  V 

8.  Orthophotos  from 
historical  photos 

Average  point  location 
offset  errors 

<  10  m 

2.3  m  V 

9.  DEMs  from  historical 
photos 

Percent  correct  detection 
of  craters 

Improved  crater 
detection 

No  improvement  noted 

Qualitative  Performance  Objective 

10.  Ease  of  use  and 
technology  transfer 

Digital  image  restoration  and  enhancement  techniques  relatively  straightforward  to 
implement  with  positive  improvements  in  interpretation  results;  photogrammetry  and 
stereo  viewing  hardware  and  software  are  comparatively  more  expensive  and  tedious 
to  implement.  V 

V  =  Indicates  project  success  criteria  met. 


Perfonnance  Objective  #9,  high  resolution  DEMs  for  improved  crater  detection,  was  anticipated 
to  be  exploratory  in  nature.  Upon  detailed  analysis,  the  photographs  examined  did  not  allow 
suitable  elevation  posting  density  to  identify  craters. 

The  success  criteria  for  Performance  Objective  #3,  the  identification  of  all  range  target  features, 
was  set  high  (90%)  and  nearly  met  by  the  DIP  method  (84%).  Performance  results  for  the  PI  and 


29 


DIP  methods  were  both  higher  than  the  ASR  results,  mapping  up  to  twice  as  many  range-related 
features.  The  DIP  method  proved  9%  better  than  the  PI  method,  correctly  identifying  seven 
more  features.  Three  of  these  were  faint  air-to-air  gunnery  range  (ATAGR)  markers  at  Site  #1, 
where  image  contrast  enhancements  and  sharpening  proved  useful.  The  remaining  five  features 
included  two  bridge  or  convoy  targets  at  Site  #3,  a  ship  target  at  Site  #4,  and  an  HE  target  circle 
at  Site  #8.  The  use  of  older  dates  and  better  resolutions  of  photography,  however,  allowed  the 
validation  of  an  additional  13  range  features.  The  availability  of  these  older  dates  of 
photography  was  considered  the  most  significant  factor  for  the  validations. 

A  majority  of  the  interpretation  omissions  occurred  at  Site  #3  (Kirtland  -  West  Mesa),  where  the 
1967  time  frame  of  the  photography  was  not  suitable  for  identifying  many  of  the  features. 
Similar  problems  were  encountered  at  Site  #2  and  Site  #5,  where  the  earliest  dates  of 
photography  used  for  the  ASR  investigations  were  1974  and  1971,  respectively.  Less  common 
as  a  limiting  factor  was  photography  scale;  the  DIP  results  for  1954  photography  of  Site  #8 
(scale  1:60,000)  missed  two  features  that  were  distinct  on  better  resolution  (scale  1:20,000) 
photography  acquired  just  one  year  earlier  (1953). 

The  success  criteria  for  Objective  5,  the  identification  of  ranges  with  craters,  was  set  at  75%. 
Overall,  the  results  for  identification  of  craters  were  comparatively  similar  among  the  methods. 
Both  the  ASR  and  DIP  approaches  found  7  of  10  areas  with  craters,  which  at  70%  is  below  the 
success  criteria  of  75%.  The  DIP  and  PI  approach  did  not  identify  three  crater  sites  on 
Guadalupe,  but  the  PI  approach  identified  one  additional  crater  area  on  another  site  (Site  #5)  and 
the  DIP  approach  identified  additional  crater  areas  on  two  other  sites  (Sites  #5  and  #8). 

Based  on  prior  review  of  ASR  documentation,  it  was  anticipated  that  the  HE  ranges  would  be 
more  readily  distinguished  by  the  presence  of  numerous  craters.  This  proved  true  for  the  four 
heavily  used  demolition  ranges  examined  in  this  project.  However,  several  instances  of  very 
limited  HE  use  occurred  on  ranges  found  on  Site  #1  (Guadalupe).  Documentation  indicates  that 
some  practice  ranges  had  limited  HE  usage  for  secure  (classified)  testing  that  may  have  involved 
only  a  small  number  of  HE  bombs. 

Site  #7  (Midland)  presented  another  limited  HE  range  situation.  According  to  ASR  (2000) 
documentation,  the  Anny  Air  Corps  invited  local  residents  to  a  July  4th,  1944,  demonstration  air 
show  that  involved  the  use  of  HE  bombs  over  a  formation  of  vehicles,  the  remnants  of  which 
were  noted  during  field  investigations.  The  relatively  uniform  landscape  conditions  of  this  site 
and  availability  of  timely  (1946)  and  excellent  scale  (1:20,000)  aerial  photography  allowed  ready 
identification  of  the  HE  craters.  The  more  variable  landscape  conditions  and  later  (1950) 
photography  at  a  smaller  scale  (1:44,000)  that  was  used  for  Site  #1  (Guadalupe)  made  similar 
identifications  more  problematic. 
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9.0  COST  ASSESSMENT 


Cost  information  for  the  demonstration  project  was  tracked  for  several  key  activities.  A 
substantial  amount  of  prior  experience  on  similar  historical  PI  projects  was  also  available  and 
reviewed.  Based  on  this  infonnation  a  basic  cost  model  was  developed  for  operational  costs  of 
the  recommended  DIP  approach.  Capital  costs  were  also  collected  for  the  necessary  hardware 
and  software  needed.  Potential  training  costs,  however,  are  not  addressed. 

9.1  COST  MODEL 

Table  6  summarizes  estimated  operating  costs  of  Pis  for  two  site  sizes: 

•  1  square  mile  sites,  such  as  Site  #6  (Kirtland  AFB  PBR  #18  Target  S-5) 

•  20  square  mile  sites,  similar  to  Sites  #3  (Kirtland  -  West  Mesa)  and  #8  (Dalhart) 

Table  6.  Operating  cost  model  for  digital  image  processing  and  interpretations. 


Cost  Element 

Data  Tracked  During 
Demonstration 

Estimated  Costs 
Small  Site 

1  Square  Mile 
(640  acres) 

(250  hectares) 

Estimated  Costs 
Medium  Site 

20  Square  Miles 
(12,800  acres) 

(5,180  hectares) 

Archive  search  and  data 
management 

-  Hours  required 

-  Personnel  required 

-  NARA  search  subcontract 

$1150 

$1150 

Photo  scanning 

-  Cost  per  frame 

-  Number  of  frames 

$900 

$1800 

Pre-processing  of 
imagery 

-  Hours  required 

-  Personnel  required 

$1300 

$2340 

Image  interpretation 
and  review 

-  Hours  per  stereo  pair 

-  Number  of  stereo  pairs 

-  Personnel  required 

$1040 

$1820 

Data  compilation  and 

QC 

-  Time  required 

-  Personnel  required 

$260 

$520 

Project  management 

-  Time  required 

-  Personnel  required 

$640 

$640 

TOTAL  COST  PER  SITE 

$5290 

$8270 

Cost  per  hectare 
(Cost  per  acre) 

$21.16  per  hectare 
($8.27  per  acre) 

$1.60  per  hectare 
($0.65  per  acre) 

It  was  assumed  that  a  minimum  of  three  dates  of  stereo  photography  would  be  acquired  for  each 
site.  Additional  dates  of  photography  would  increase  costs  proportionately.  The  need  for 
additional  dates  should  be  based  on  site-specific  conditions.  It  was  also  assumed  that  one 
historical  orthophoto  would  be  prepared  and  used  as  a  base  for  mapping  interpretation  results. 
Simpler  georeferencing  techniques  were  assumed  for  the  other  dates. 

The  historical  photo  scales  examined  for  this  project  ranged  from  1:20,000  to  1:63,000.  At  the 
preferred  1:20,000  scale,  each  frame  covers  about  8  square  miles  (2,072  ha).  Each  frame  at 
1:63,000  scale  covers  about  80  square  miles  (20,719  ha).  Smaller  scale  photography  (e.g., 


31 


1:63,000)  can  sometimes  prove  inadequate  for  small  features,  but  may  still  be  useful  if  it  is  the 
only  photography  available  for  a  critical  time  frame. 

Figure  10  provides  an  example  of  1:20,000  versus  1:60,000  scale  photos  for  Site  #8  (Dalhart). 
The  2009  SI  made  use  of  the  1:60,000  scale  photos  (1954)  and  did  not  identify  the  FIE  range 
location.  The  PI  and  DIP  interpretations  both  identified  a  probable  HE  range  in  this  area  from 
the  1954  photos.  The  1:20,000  scale  photos  (1953)  were  used  to  validate  the  HE  range  and  also 
allowed  two  additional  range  features  to  be  identified. 


Figure  10.  Original  photo  scale  comparison. 

Difference  between  the  original  photo  scales  results  in  substantially  better  target  detection  on  the  larger  scale 
(1:20,000)  photos.  Contour  farming  practice  at  this  site  made  interpretations  difficult.  The  ship  target  feature  was 
not  initially  validated  on  the  1953  photos  until  a  target  layout  map  included  in  the  ASR  documentation  was 
reviewed.  The  layout  map  included  target  shapes,  sizes,  and  distances  between  the  three  targets,  but  incorrectly 
located  the  series  of  targets  as  being  in  the  middle  of  the  site,  about  one  third  of  a  mile  west  of  their  actual  location. 

The  2009  SI  used  the  1954  photography  but  did  not  identify  the  HE  range  and  craters  or  the  other  targets  noted 
above.  The  PI  and  DIP  analysts  correctly  identified  the  HE  range  and  craters  on  the  1954  photographs  but  missed 

the  other  two  targets. 


32 


In  addition  to  scale,  the  specific  number  of  frames  required  for  stereo  coverage  of  a  site  also 
varies  by  the  alignment  of  the  photo  mission  flight  lines  and  camera  stations.  Film  acquisition, 
scanning,  preprocessing,  and  some  other  costs  do  not  become  proportionally  linear  until  site 
sizes  are  larger  than  the  area  covered  by  individual  frames  of  photography.  As  shown  in  Table  6, 
the  costs  per  unit  area  are  estimated  to  be  substantially  lower  for  the  20-square  mile  site  versus 
the  1 -square  mile  site.  Sites  larger  than  20  square  miles  would  scale  in  a  more  linear  fashion, 
especially  for  larger  scale  photos. 

The  operating  cost  model  does  not  include  costs  to  develop  a  report  documenting  interpretation 
results.  Nor  are  costs  to  develop  appropriate  training  keys  and  train  personnel  in  procedures 
included.  Training  costs  can  vary  significantly  depending  upon  the  prior  qualifications  of  the 
personnel  involved. 

Capital  cost  estimates  for  appropriate  hardware  and  software  are  summarized  as  follows: 


High  capacity  computer  workstation  $5000 

Stereo  viewing  display  monitor  $4500 

Image  processing  software  with  3-D  photogrammetry  modules  $8500 

Basic  GIS  software  package  $1500 


Total  $19,500 

A  high  capacity  workstation  is  recommended  due  to  the  substantial  image  file  sizes  and 
processing  requirements  for  advanced  image  processing  algorithms.  Such  workstations  have 
multipurpose  uses  and  are  relatively  common  for  technical  analysts.  Appropriate  image 
processing  and  GIS  software  are  also  relatively  standard  and  have  multipurpose  uses.  The  stereo 
viewing  display  monitors  that  are  integrated  with  specific  image  processing  and  photogrammetry 
software,  however,  are  relatively  specialized. 

9.2  COST  DRIVERS 

As  indicated  by  Table  6,  the  primary  cost  drivers  for  historical  Pis  are  related  to  photo  searches, 
scanning  and  preprocessing  of  digital  imagery,  and  the  image  analyses.  The  results  of  this 
demonstration  project  indicate  that  the  historical  documentation  and  aerial  photography  used  for 
existing  range  definitions  should  always  be  reviewed  to  ensure  their  suitability. 

9.3  COST  BENEFIT 

Earlier  Pis  of  2005  digital  orthophotos  to  support  the  New  Mexico  statewide  FUDS  GIS  database 
development  identified  several  mislocated  ranges  and  additional  range  target  features.  These  Pis 
helped  focus  SI  efforts  and  avoided  potentially  costly  rework  for  several  MMRP  sites.  The 
results  of  the  Pis  greatly  reduced  the  required  area  to  be  traversed  and  sampled  during  the  SI 
field  efforts,  resulting  in  a  much  more  satisfied  regulatory  community  as  pertains  to  the 
attainment  of  the  Data  Quality  Objectives  in  the  MMRP  SI  program.  As  demonstrated  in  this 
project,  a  more  systematic  review  and  use  of  historical  photography  can  potentially  provide  even 
more  benefits. 
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A  detailed  cost  benefit  assessment  was  not  possible  for  this  demonstration.  Several  missing  or 
mislocated  range  target  features  were  observed  within  the  selected  study  areas  demonstration 
project.  These  features  included  an  HE  range  that  was  not  identified  in  a  recent  2009  SI  (Site  #8, 
Dalhart).  In  addition,  five  distinct  bombing  ranges  that  do  not  appear  in  the  current  FUDS 
MMRP  inventory,  and  not  related  to  the  selected  study  sites,  were  identified  during  the  course  of 
photo  reviews  for  this  project.  The  locations  for  these  sites  have  been  forwarded  to  the  USAGE 
for  a  review  of  their  potential  FUDS  status. 
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10.0  IMPLEMENTATION  ISSUES 


The  use  of  film  diapositives  instead  of  prints  for  historical  Pis  is  a  relatively  straightforward  step 
with  only  modest  cost  impacts  for  specialized  equipment.  Much  of  the  benefits  can  most  likely 
be  obtained  using  intermediate  level  equipment  (light  boxes  and  mirror  stereoscopes  with 
magnifying  optics  in  lieu  of  more  expensive  stereo  zoom  equipment).  However,  photographic 
film  archives  are  steadily  moving  towards  a  digital  future  that  already  requires  the  use  of  DIP  for 
photos  from  some  primary  archive  sources. 

Digital  image  products  have  also  become  a  standard  element  of  GIS  databases.  This  project 
demonstrated  that  DIP  results  generally  surpassed  optical  PI  results.  Implementation  of  DIP  for 
improved  analysis  of  historical  aerial  photographs  is  therefore  recommended  as  the  preferred 
method  of  analysis  for  this  application.  Based  on  this  and  similar  projects,  some  basic  general 
guidelines  for  improved  use  of  historical  photography  are  summarized  below: 

•  Collect  available  archive  data  and  define  expectations  for  the  site: 

-  Previous  reports  often  provide  substantial  insights  about  a  site’s  history 
and  what  type  of  features  are  to  be  expected. 

-  Analysts  need  to  be  flexible  and  open  to  finding  unexpected  features,  but 
blind  PI  unnecessarily  hinders  efficient  analyses. 

•  Review  available  analysis  results,  such  as: 

-  Visibility  of  distinct  features  on  available  photos. 

-  Time  frame  of  available  photos,  ideally  from  the  operating  period  or 
shortly  thereafter.  If  needed,  earlier  photography  can  sometimes  help 
identify  pre-existing  features. 

-  Suitable  resolution  (primarily  scale)  to  see  detail;  a  scale  range  of  1 : 15,000 
to  1:25,000  is  preferred,  when  available,  although  scales  up  to  1:60,000 
have  been  found  useful  when  better  scales  are  not  available. 

-  Stability  of  landscape — sandy  and  windy  environments  are  likely  to  more 
rapidly  obscure  range  features,  as  can  agriculture  or  urban  development. 

•  Conduct  extended  photo  search  (if  needed): 

-  Review  national  and  regional  photo  sources. 

Recommend  pre-activity  period  and  post-activity  photos;  decade  intervals 
may  be  adequate  for  post-activity  review  (to  avoid  unrelated  false  alanns). 

•  Acquire  diapositive  copies  or  digital  scans  (preferred)  from  original  source 

materials. 

•  Preprocess  imagery  for  basic  brightness  and  contrast,  blur,  noise,  and  edge 

enhancements. 
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•  Georeference  all  photos  (minimally)  and  create  orthophotos  if  terrain  conditions 
are  a  significant  factor  (i.e.,  hilly  or  mountainous). 

•  Develop  a  set  of  training  keys  with  descriptions. 

•  Develop  a  suitable  classification  scheme: 

-  Include  confidence  criteria  (e.g.,  confident,  probable,  possible). 

-  Include  open  AOI  for  additional  items,  with  comments. 

-  Map  transportation  features  (roads  and  trails). 

•  Monoscopic  viewing  may  be  adequate  for  some  features,  but  stereo  viewing  is 
recommended  and  sometimes  required. 

•  Use  a  GIS  data  collection  framework  to  record  and  attribute  interpretation 
features. 

This  approach  allows  other  datasets  to  be  viewed  in  context  and  supports  the  “convergence  of 
evidence”  required  for  some  interpretations. 

An  extensive  and  growing  set  of  historical  aerial  photo  archives  exist.  These  archives  are 
becoming  more  readily  searchable  with  improved  finding  aids.  The  need  to  conduct  updated 
historical  photo  searches  can  be  critical  to  the  identification  of  appropriate  dates  and  scales  of 
photography  for  FUDS  MMRP  assessments. 

The  trend  to  digital  scanning  of  these  archives  is  clear,  and  DIP  has  become  a  requirement  for 
using  some  sources  of  imagery.  Similarly,  the  use  of  historical  photos  in  a  GIS  framework, 
which  can  facilitate  interpretations,  requires  digital  imagery.  As  demonstrated  by  this  project, 
the  interpretation  of  digitally  processed  imagery  provided  results  that  were  systematically  similar 
or  better  than  analog  film  interpretations. 
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